r? @epage

rustbot has assigned @epage.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

❗ Help wanted: benchmark on Windows and HDD ❗

Benchmark result

• commit: 58655e5
• project: awslabs/aws-sdk-rust@f849309

Cache write

Result: naive SQLite cache is significant slower than fs cache on macOS and Linux. This might be the index cache blobs are too big. There is a post on SQLite website about storing blobs inside DB file versus separate files https://www.sqlite.org/intern-v-extern-blob.html. The rule of thumb from that experiment is file size larger than 100k would be better to store separately. I've picked some popular crates, and most of them are over 50k.

size-in-kb,crate
15,sha1
19,memchr
23,base64
33,walkdir
35,tar
36,log
42,sha2
42,thiserror
44,http
54,anyhow
55,libc
59,proc-macro2
63,tracing
67,url
80,rand
92,futures
131,serde
160,tonic
174,openssl
175,regex
398,windows-sys
460,syn
490,reqwest
568,tokio
572,cargo
603,hyper
845,rustix

benchmark script

hyperfine --warmup 2 \
  --min-runs 100 \
  -L ZFLAG -Zindex-cache-sqlite, \
  --setup "CARGO_REGISTRIES_CRATES_IO_PROTOCOL=git CARGO_HOME='$MY_CARGO_HOME' '$MY_CARGO' fetch" \
  --prepare 'rm -rf "$MY_CARGO_HOME/registry/index/<git-index-url-and-hash>/.cache"' \
  "CARGO_REGISTRIES_CRATES_IO_PROTOCOL=git CARGO_HOME='$MY_CARGO_HOME' '$MY_CARGO' generate-lockfile --offline {ZFLAG}"

AWS EC2 c6a.8xlarge
vCPU: 32
host: x86_64-unknown-linux-gnu
os: Amazon Linux AMI 2.0.0 [64-bit]
disk: AWS EBS SSD 500GB (NVMe)

Apple Macbook M1 Pro
CPU: 10 (8 + 2)
mem: 32 GB
host: aarch64-apple-darwin
os: Mac OS 14.3.1 [64-bit]

Cache read

Result: no significant different between Linux and macOS on cache read

benchmark script

hyperfine --warmup 2 \
  --min-runs 100 \
  -L ZFLAG -Zindex-cache-sqlite, \
  --setup "CARGO_HOME='$MY_CARGO_HOME' '$MY_CARGO' generate-lockfile {ZFLAG}" \
  "CARGO_HOME='$MY_CARGO_HOME' '$MY_CARGO' generate-lockfile --offline {ZFLAG}"

AWS EC2 c6a.8xlarge
vCPU: 32
host: x86_64-unknown-linux-gnu
os: Amazon Linux AMI 2.0.0 [64-bit]
disk: AWS EBS SSD 500GB (NVMe)

Apple Macbook M1 Pro
CPU: 10 (8 + 2)
mem: 32 GB
host: aarch64-apple-darwin
os: Mac OS 14.3.1 [64-bit]

Disk usage

• Linux
  • SQLite cache: 17,555,456
  • old fs cache: 17,303,091
• macOS
  • SQLite cache: 17,555,456
  • old fs cache: 17,303,091
• Windows

One of the things that was important for the last-use performance was to batch the inserts in a single transaction. From what I can tell, this is using a separate transaction for each insert which can be expensive. That might be something to experiment with.

Another thought I had was that the schema could be (name, version, blob), so that the blob is only stored per index line which should be a lot smaller than the entire blob of all versions. That would require some deeper changes of course. It might also be tricky to avoid too much JSON parsing overhead to reliably extract the version.

Have you also tried adjusting the page size?

Yes, both are good ideas, though requires more than trivial changes and is hard to share the same interface with old cache mechanism 😞.

Have you also tried adjusting the page size?

Yes. And also a series of combination of WAL journal, cache size/limit, synchronouns=normal, and other pragmas. The difference was insignificant.

☔ The latest upstream changes (presumably #13632) made this pull request unmergeable. Please resolve the merge conflicts.

batch the inserts in a single transaction. From what I can tell, this is using a separate transaction for each insert which can be expensive. That might be something to experiment with.

I've pushed some variants of this:

• deferred: 4b4a993 — defer all insertions until dropping CacheManager.
• One-JSON-per-row: aadab5f — deferred + change the schema to (name, version, one line of JSON blob)
• blob dump: 4ab7aab — this is the initial "dump" everything to db" integration
• old fs: 3b43aa8

On macOS, both of the read and write performance of these two variants are on par with the original filesystem cache.

On Linux, however, we've got a 5% performance hit on write.

@ehuss do you have time doing a simple benchmark on Windows?

I also wonder if it is not worthy due to

• the complexity we'll introduce
• deferring insertion until drop seems too hacky
• the database solution might not scale well if blobs become too large.

Going to close this as it is not going to anywhere in a near future. We've collected some interesting data points and people can look into them when needed :)